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USER DEFINABLE IMAGE REFERENCE POINTS 



CROSS-REFERENCE TO RELATED APPLICATIONS 

The present Application claims the benefit of United States Provisional Patent 
Application No. 60/336,498 titled "User Definable Image Reference Points" filed October 24, 
2001, the content of which is incorporated by reference in this disclosure in its entirety. 

BACKGROUND 

The present invention relates to an application program interface and methods for 
combining any number of arbitrarily selected image modifications in an image while 
assigning those modifications easily to an image area and/or image color to provide for 
optimally adjusting color, contrast, sharpness, and other image-editing functions in the 
image-editing process. 

It is a well-known problem to correct color, contrast, sharpness, or other specific 
digital image attributes in a digital image. It is also well-known to those skilled in image- 
editing that it is difficult to perform multiple color, contrast, and other adjustments while 
maintaining a natural appearance of the digital image. 

At the current stage of image-editing technology, computer users can only apply 
relatively basic functions to images in a single action, such as increasing the saturation of all 
pixels of an image, removing a certain colorcast from the entire image, or increasing the 
image's overall contrast. Well-known image-editing tools and techniques such as layer 
masks can be combined with existing image adjustment functions to apply such image 
changes selectively. However, current methods for image editing are still limited to one 
single image adjustment at a time. More complex tools such as the Curves functions 
provided in image r edi ting programs such as Adobe Photoshop® provide the user with added 
control for changing image color, but such tools are difficult to apply, and still very limited as 
they apply an image enhancement globally to the image. 

Additional image editing tools also exist for reading or measuring color values in the 
digital image. In its current release, Adobe Photoshop® offers a feature that enables the user 
to place and move up to four color reference points in an image. Such color reference points 
read properties (limited to the color values) of the image area in which they are placed. It is 
known to those skilled in the art that the only purpose of such color reference points is to 
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display the associated color values; there is no image operation associated with such 
reference points. The reference points utilized in image-editing software are merely offered 
as a control tool for measuring an image's color values at a specific point within the image. 
In other implementations of reference points used for measuring color in specific 
5 image regions, image-editing applications such as Adobe Photoshop®, Corel Draw®, and 
Pictographies iCorrect 3.0®, allow the user to select a color in the image by clicking on a 
specific image point during a color enhancement and perform an operation on the specific 
color with which the selected point is associated. For example, the black-point adjustment in 
Adobe Photoshop® allows the user to select a color in the image and specify the selected 
10 color as black, instructing the software to apply a uniform color operation to all pixels of the 
image, so that the desired color is turned into black. This method is not only available for 
black-point operations, but for pixels that are intended to be white, gray (neutral), skin tone, 
or sky, etc. 

While each of these software applications provide methods for reading a limited 
1 5 number of colors and allow for one single operation which is applied globally and uniformly 
to the image and which only applies one uniform color cast change based on the read 
information, none of the methods currently used allow for the placement of one or more 
graphical representations of image reference points (IRPs) in the image that can read color or 
image information, be assigned an image editing function, be associated with one or more 
20 image reference points (IRPs) in the image to perform image-editing functions, be moved, or 
be modified by the user such that multiple related and unrelated operations can be performed. 

What is needed is an application program interface and methods for editing digital 
images that enable the user to place multiple, arbitrary reference points in a digital image and 
assign image-editing functions, weighted values or any such combinations to enable multiple 
25 image-editing functions to be applied to an image. 

SUMMARY 

The present invention meets this need by enabling users to perform such complex 
image-editing operations easily by performing multiple, complex image enhancements in a 
single step. A method is described that allows the user to place a plurality of Image Reference 
30 Points [IRPs] in a digital image, assign an image-editing function to each IRP, and alter each 
image-editing function based on the desired intensity, effect, and its effect relative to other 
IRPs placed in the image, via any one of a variety of interface concepts described later in this 
disclosure. 
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A method for image processing of a digital image is disclosed comprising the steps of 
determining one or more sets of pixel characteristics; determining for each pixel 
characteristic set, an image editing function; providing a mixing function algorithm embodied 
on a computer-readable medium for modifying the digital image; and processing the digital 
5 image by applying the mixing function algorithm based on the one or more pixel 

characteristic sets and determined image editing functions. In one embodiment, the mixing 
function algorithm comprises a difference function. Optionally, the difference function 
algorithm calculates a value based on the difference of between pixel characteristics and one 
of the one or more determined pixel characteristic sets. In another embodiment, the mixing 

10 function algorithm includes a controlling function for normalizing the calculations. 

In a further embodiment, the method adds the step of determining for each pixel 
characteristic set, a set of weighting values, and the processing step further comprises 
applying the mixing function algorithm based on the determined weighting value set. 

In a further embodiment, a first pixel characteristic set is determined, and at least one 

1 5 characteristic in the first pixel characteristic set is location dependent, and at least one 
characteristic in the first pixel characteristic set is either color dependent, or structure 
dependent, or both. Alternatively, a first pixel characteristic set is determined, and at least 
two different characteristics in the first pixel characteristic set are from the group consisting 
of location dependent, color dependent, and structure dependent. 

20 A method for processing of a digital image comprising the steps of receiving the 

coordinates of one or more than one image reference point defined by a user within the digital 
image; receiving one or more than one image editing function assigned by the user and 
associated with the coordinates of the one or more than one defined image reference point; 
providing a mixing function algorithm embodied on a computer-readable medium for 

25 modifying the digital image; and processing the digital image by applying the mixing 

function algorithm based on the one or more than one assigned image editing function and 
the coordinates of the one or more than one defined image reference point. 

The method may optionally further comprise displaying a graphical icon at the 
coordinates of a defined image reference point. 

30 A mixing function algorithm suitable to the invention is described, and exemplar 

alternative embodiments are disclosed, including a group consisting of a Pythagoras distance 
approach which calculates a geometric distance between each pixel of the digital image to the 
coordinates of the one or more than one defined image reference point, a color curves 
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approach, a segmentation approach, a classification approach, an expanding areas approach, 
and an offset vector approach. Optionally, the segmentation approach comprises multiple 

v. segmentation, and additionally optionally the classification approach adjusts for similarity of 
pixel attributes. The mixing function algorithm may optionally operate as a function of the 
5 calculated geometric distance from each pixel of the digital image to the coordinates of the 
defined image reference points. 

Optionally, the disclosed method further comprises receiving one or more assigned 
image characteristics associated with the coordinates of a defined image reference point, and 
wherein the mixing function algorithm calculates a characteristic difference between the 

10 image characteristics of a pixel of the digital image and the assigned image characteristics. 
The mixing function algorithm may also calculate a characteristic difference between the 
image characteristics of a pixel and the image characteristics of one or more pixels 
neighboring the coordinates of one or more defined image reference point. 

Additionally, optionally other steps may be added to the method. For example, the 

1 5 method may further comprise receiving one or more weighting values, and the processing 

step further comprising applying the mixing function algorithm based on weighting values; or 
further comprise receiving one or more regions of interest associated with the coordinates of 
one or more defined image reference point; or further comprise the step of providing an 
application program interface comprising a first interface to receive the coordinates of the 

20 one or more defined image reference points, and a second interface to receive the one or more 
assigned image editing functions. 

A method for processing of a digital image comprising pixels having image 
characteristics is disclosed comprising the steps defining the location of image reference 
points within the digital image; determining image editing functions; and processing the 

25 digital image by applying the determined image editing functions based upon either the 

location of the defined image reference points, or the image characteristics of the pixels at the 
location of the defined image reference points, or both. 

A method for image processing of a digital image is also disclosed comprising the 
steps of providing one or more than one image processing filter; setting the coordinates of 

30 one or more than one image reference point within the digital image; providing a mixing 
function algorithm embodied on a computer-readable medium for modifying the digital 
image; and processing the digital image by applying the mixing algorithm based on the one 
or more than one image processing filter and the coordinates of the one or more than one set 
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image reference point. Optionally, various filters may be used, including but not limited to a 
noise reduction filter, a sharpening filter, or a color change filter. 

An application program interface is provided embodied on a computer-readable 
medium for execution on a computer for image processing of a digital image, the digital 
5 image comprising pixels having image characteristics, comprising a first interface to receive 
the coordinates of each of a plurality of image reference points defined by a user within the 
digital image, and a second interface to receive an image editing function assigned by the 
user and associated with either the coordinates of each of the plurality of defined image 
reference points, or the image characteristics of one or more pixels neighboring the 

10 coordinates of each of the plurality of defined image reference points. 

In a further embodiment, the second interface is to receive an image editing function 
assigned by the user and associated with both the coordinates of each of the plurality of 
defined image reference points, and the image characteristics of one or more pixels 
neighboring the coordinates of each of the plurality of defined image reference points. 

15 In a further alternative optional embodiment, the program interface further comprises 

a third interface that displays a graphical icon at the coordinates of one or more than one of 
the plurality of defined image reference points. Additionally optionally, the third interface 
permits repositioning of the graphical icon. 

In further embodiments, the program interface further comprises a fourth interface 

20 that displays the assigned image editing function. The second interface may further receive 
an image area associated with the coordinates of one or more than one of the plurality of 
defined image reference points. The second interface may further receive a color area 
associated with the coordinates of one or more than one of the plurality of defined image 
reference points. 

25 A further embodiment of an application program interface is disclosed, embodied on a 

computer-readable medium for execution on a computer for image processing of a digital 
image, the digital image comprising pixels having image characteristics, comprising a first 
interface to receive the coordinates of an image reference point defined by a user within the 
digital image, and a second interface to receive an image editing function assigned by the 

30 user and associated with both the coordinates of the defined image reference point, and the 
image characteristics of one or more pixels neighboring the coordinates of the defined image 
reference point. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features, aspects, and advantages of the present invention will 
become better understood with reference to the following description, illustrations, equations, 
appended claims, and accompanying drawings where: 
5 Figure 1 is a screen shot of a digital image in an image processing program, 

illustrating one embodiment useable in the application program interface of the present 
invention. 

Figure 2 is a screen shot of a digital image in an image processing program, 
illustrating another embodiment useable in the application program interface of the present 
10 invention. 

Figure 3 is a flow chart of the steps of the application of a mixing function in accord 
with the disclosure. 

Figure 4 is an illustration of one embodiment of a dialog box useable in the 
application program interface of the present invention. 

15 Figure 5 is an illustration of one embodiment of a dialog box implementing simplified 

user control over weights useable in the application program interface of the present 
invention. 

DETAILED DESCRIPTION 

The method and program interface of the present invention is useable as a plug-in 
20 supplemental program, as an independent module that may be integrated into any 

commercially available image processing program such as Adobe Photoshop®, or into any 
image processing device that is capable of modifying and displaying an image, such as a 
color copier or a self service photo print kiosk, as a dynamic library file or similar module 
that may be implemented into other software programs whereby image measurement and 
25 modification may be useful, or as a stand alone software program. These are all examples, 
without limitation, of image processing of a digital image. Although embodiments of the 
invention which adjust color, contrast,~noise reduction, and sharpening are described, the 
present invention is useful for altering any attribute or feature of the digital image. 

Furthermore, it will become clear with regard to the current invention that the user 
30 interface for the current invention may have various embodiments, which will become clear 
later in this disclosure. 
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The Application Program Interface 

The user interface component of the present invention provides methods for setting 
IRPs in an image. Those skilled in the art will find that multiple methods or implementations 
of a user interface are useful with regard to the current invention. 
5 In one preferred embodiment of a user interface, an implementation of the present 

invention allows the user to set a variety of types of IRPs in an image, which can be shown as 
graphic tags 10 floating over the image, as shown in Figure 1. Figure 1 is a screen shot of a 
digital image in an image processing program. 

This method enables the user to move the IRPs in the image for the purpose of 
10 adjusting the location of such IRPs and thus the effect of each IRP on the image. 

In another preferred embodiment, IRPs could be invisible within the preview area of 
the image and identified placed elsewhere as information boxes 12 within the interface, as 
shown in Figure 2, but associated with a location (shown by arrow). In this embodiment of 
the user interface, graphic tags 10 do not "float" over the image as in Figure 1. However, as 
1 5 it will become clear later in this disclosure that it is the location that Image Reference Points 
[IRPs] identifies and the related function that are significant, and that the graphical 
representations of the IRPs are useful as a convenience to the user to indicate the location of 
the IRP function. (Figure 2 is a screen shot of a digital image in an image processing 
program.) 

20 In both Figure 1 and Figure 2, the IRPs serve as a graphical representation of an 

image modification that will be applied to an area of the image. 

The application program interface is embodied on a computer-readable medium for 
execution on a computer for image processing of a digital image. A first interface receives 
the coordinates of each of a plurality of image reference points defined by a user within the 
25 digital image, and a second interface receives an image editing function assigned by the user 
and associated with either the coordinates of each of the plurality of defined image reference 
points, or the image characteristics of one or more pixels neighboring the coordinates of each 
of the plurality of defined image reference points. 

In a further embodiment, the second interface receives an image editing function 
30 assigned by the user and associated with both the coordinates of each of the plurality of 
defined image reference points, and the image characteristics of one or more pixels 
neighboring the coordinates of each of the plurality of defined image reference points. 

In a further alternative optional embodiment, a third interface displays a graphical 
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icon or graphical tag 10 at the coordinates of one or more than one of the plurality of defined 
image reference points. Additionally optionally, the third interface permits repositioning of 
the graphical icon. 

In further embodiments, a fourth interface displays the assigned image editing 
5 function. The second interface may further receive an image area associated with the 

coordinates of one or more than one of the plurality of defined image reference points. The 
second interface may further receive a color area associated with the coordinates of one or 
more than one of the plurality of defined image reference points. 

In an alternative embodiment, the first interface receives the coordinates of a single 
10 image reference point defined by a user within the digital image, and the second interface 
receives an image editing function assigned by the user and associated with both the 
coordinates of the defined image reference point, and the image characteristics of one or 
more pixels neighboring the coordinates of the defined image reference point. 
Mixing Functions 

1 5 A central function of the present invention is the "Mixing Function," which modifies 

the image based on the values and settings of the IRPs and the image modifications 
associated with the IRPs. With reference to this disclosure, a "Mixing Function" is an 
algorithm that defines to what extent a pixel is modified by each of the IRPs and its related 
image modification function. 

20 It will be evident to those skilled in the art that there are many possible mixing 

functions, as will be shown in this disclosure. 

The method for applying the mixing function is shown in Figure 3. Begin with 
receiving 14 the IRPs in the image; test 16 to determine whether abstract IRPs are being used. 
If so, load 1 8 the abstract IRPs and then select 20 the first pixel to be processed; if not select 

25 20 the first pixel to be processed. Then apply 22 the mixing function according to this 

disclosure, and test 24 whether all pixels chosen to be processed have been processed. If so, 
the method is completed 26, if not, the next pixel is selected 28 and step 22 is repeated. 
Using the Pythagoras Distance Approach 

In one embodiment of the mixing function, the Pythagoras equation can be used. 

30 Those skilled in the art will find that this is more suitable for IRPs that are intended to 
perform local color correction or similar changes to an image. 

In step 22, apply the image modification to a greater extent, if the location of the IRP 
is close to that of the current pixel, or apply it to a lesser extent, if the location of the IRP is 
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further away from the current pixel, using the Pythagoras equation to measure the distance, 
often also referred to as distance in Euclidian space. 
Using Color Curves 

In another embodiment, a mixing function could be created with the use of color 
5 curves. To create the function: 

Step 22. 1 . 1 . Begin with the first channel of the image (such as the Red channel). 

Step 22.1.2. All IRPs will have an existing brightness which is the brightness of the 
actual channel of the pixel where the IRP is located, and a desired brightness, which is the 
brightness of the actual channel of the same pixel after the image modification associated 
10 with its IRP has been applied. Find the optimal polynomial function that matches these 
values. For example, if the red channel has an IRP on a pixel with a value of 20, which 
changes the pixel's value to 5, and there is a second IRP above a pixel with the value of 80, 
which changes that channel luminosity to 90, all that is needed is to find a function that meets 
the conditions/20)=5 and^(80) = 90. 
1 5 Step 22. 1.3. Apply this function to all pixels of the selected channel. 

Step 22.1 .4. If all channels have not been modified, select the next channel and 
proceed with step 22. 1 .2. 

Using Segmentation to Create the Mixing Function 

In a further embodiment, the mixing function can be created using segmentation. To 
20 create the function: 

Step 22.2.1. Segment the image using any appropriate segmentation algorithm. 
Step 22.2.2. Begin with IRP 1. 

Step 22.2.3. Apply the filter associated with that IRP to the segment where it is 
located. 

25 Step 22.2.4. Select the next IRP. 

Step 22.2.5. Unless all IRPs have been processed, proceed with step 22.2.3. 
If there is a segment that contains two IRPs, re-segment the image with smaller 
segments, or re-segment the area into smaller segments. 
Using Multiple Segmentations 
30 In a still further embodiment of the current invention, the mixing function can be 

created using multiple segmentation. To create the function: 

Step 22.3.1. Make "n" different segmentations of the image, e.g., n=4, where the first 
segmentation is rougher, (having few but larger segments), and the following segmentations 
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are finer, (using more by smaller segments per image). 
Step 22.3.2. Begin with IRP 1. 

Step 22.3.3. Apply the image modification of that IRP at 1/nth opacity to all pixels in 
the segment that contains the current IRP of the first segmentation, then apply the image 
5 modification at 1/nth opacity to all pixels in the segment containing the IRP of the second 
segmentation. Continue for all n segmentations. 

Step 22.3.4. Select the next IRP. 

Step 22.3.5. Unless all IRPs have been processed, proceed with step 22.3.3. 

Those skilled in the art will know that several segmenting algorithms may be used, 
10 and the "roughness" (size of segments) within the equation can be defined by a parameter. 
Using a Classification Method 

A classification method from pattern recognition science may be used to create 
another embodiment of the mixing function. To create the function: 

Step 22.4.1. Choose a set of characteristics, such as saturation, x-coordinate, y- 
1 5 coordinate, hue, and luminance. 

Step 22.4.2. Using existing methods of pattern recognition, classify all pixels of the 
image, i.e., every pixel is assigned to an IRP based on the characteristics, and assuming that 
the IRPs are centers of clusters. 

Step 22.4.3. Modify each pixel with the image modification associated with the IRP 
20 to which the pixel has been classified. 
Using a "Soft" Classification Method 

In an even> further embodiment of the current invention, it may be useful to modify the 
classification method to adjust for similarity of pixel attributes. 

Typically, a pixel will not match the attributes of one IRP to a degree of 100%. One 
25 pixel's attributes might, for example, match one IRP to 50%, another IRP to 30% and a third 
IRP only to 20%. In the current embodiment using soft classification, the algorithm would 
apply the effect of the first IRP to a degree of 50%, the second IRP's effect at 30%, and the 
third IRP's effect to 20%. By utilizing this "Soft" Classification, one pixel is not purely 
associated with the most similar IRP. 
30 One preferred embodiment that is described in detail later in this disclosure will show " 

an implementation that follows a similar concept as described here. 
Using an Expanding Areas Method 

In another embodiment of the mixing function, an expanding areas method could be 
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used to create a mixing function. To create the function: 

Step 22.5.1. Associate each IRP with an "area" or location within the image. Initially, 
this area is only the pixel where the IRP is positioned. 

Step 22.5.2. Apply the following to all IRP areas: Consider all pixels that touch the 
5 area. Among those, find the one whose attributes (color, saturation, luminosity) are closest to 
the initial pixel of the area. While comparing the attributes, minimize for the sum of 
differences of all attributes. Add this pixel to the area and assign the current area size in 
pixels to it. The initial pixel is assigned with a value of 1, the next added pixel is assigned a 
value of 2, the next with a value of 3, etc., until each pixel has been assigned a value. 
10 Step 22.5.3. Repeat step 22.5.2 until all areas have expanded to the full image size. 

Step 22.5.4. Apply all modifications of all IRPs to that pixel while increasing the 
application for those with smaller values. 
One Preferred Mixing Function 

In one preferred embodiment, a mixing function uses a set of attributes for each pixel 
15 (luminosity, hue, etc.). These attributes are compared to the attributes of the area where an 
IRP is positioned, and the Mixing Function applies those IRPs image modifications more 
whose associated attributes are similar to the actual pixel, and those IRPs image 
modifications less whose associated characteristics are very different from the actual pixel. 

Unless otherwise specified, capitalized variables will represent large structures (such 
20 as the image I) or functions, while non-capitalized variables refer to one-dimensional, real 
numbers. 

Definition of the Key Elements 

A "Pixel-Difference-Based IRP Image Modification," from now on called an "IRP 
Image Modification," may be represented by a 7-tuple, as shown in Equation 1, where m is 
25 the amount of IRPs that will be made use of, and the number n is the amount of analyzing 
functions as explained later. 

(F^R^XA.^^V,^) [1] 

The first value, F\„ m is a set of the "Performing Functions." Each of these functions 
is an image modification function, which may be called with three parameters as shown in 
30 Equation 2. 

I' JV =F(l 9 x 9 y) [2] 
In Equation 2 the result I ^ is the pixel that has been calculated by F. I is the image 
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on which F is applied, and x andy are the coordinates of the pixel in I that F is applied to. 
Such a performing function could be "darken pixels by 30%," for example, as shown in 
Figure 1 . In image science, these modifications are often called filters. 

The second value in Equation 1, R\ mnM is a number of m tuples. Each tuple represents 
5 values of an IRP, and is a set of pixel characteristics. Such a tuple R consists of 2*h+1 
values, as in Equation [3]. 

((&-g n )>g*M-K)) [3] 
F\ ... m and R\ mmjn together represent the IRPs that the user has created. I will explain 
later how the IRPs that the user has placed can be converted into the functions and values 
10 Fi . m and R\ ... m . Later in this disclosure I indicate that a function F and a tuple R are 
"associated" with each other and with an IRP if they F and R together represent an IRP. 

The third value I in Equation 1 is the image with the pixels 1^. This image can be of 
any type, i.e., grayscale, Lab, CMYK, RGB, or any other image representation that allows 
Performing Functions (Equation [2]) or analyzing functions (Equation [4]) to be performed 
15 on the image. 

The fourth element A\., M in Equation 1 is a set of n "Analyzing Functions" as 
represented in Equation [4]. 

4(W) = * [4] 
These functions, unlike the Performing Functions F, calculate a single real number k 
20 for each pixel. These functions extract comparable attributes out of the image, such as 

saturation, luminance, horizontal location or vertical location, amount of noise in the region 
around the coordinates x, y, and so forth. The number n is the amount of Analyzing 
Functions. 

The function's results need to be comparable. That is, the difference of the results of 
25 two different pixels applied to the same Analyzing Function can be represented by a number. 
For example, if p\ is a dark pixel and pi is a bright pixel, and A is a function that calculates 
the luminance of a pixel, then \A(p\)-A(p 2 )\ is an easy measure for the luminosity difference of 
both pixels. Note: Analyzing Functions in this disclosure refer to functions that calculate 
characteristics of an image, and must not be confused with the mathematical term "analytic 
30 functions." The result of an Analyzing Function applied to a pixel will for further reference 
in this disclosure be called a "Characteristic" of the pixel. 

The Analyzing Functions can analyze the color of a point*, y in the image I, the 
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structure of the point x, y in the image I, and the location of a point x, y in the image I itself. 

Later in this disclosure I refer to "Color Analyzing Functions," "Structure Analyzing 
Functions" and "Location Analyzing Functions." Color Analyzing Functions are any 
functions on the pixel's values itself, such as r, g and 6, while Structure Analyzing Functions 
5 also take the values and differences of a group of pixel around the point x, y into account, and 
Location Analyzing Functions are any functions on x andy. 

For example, the Analyzing Function A(JL, x, y) = x + y is a Location Analyzing 
Function of the pixel. An example of a Color Analyzing Function would be A(J, x f y) = Ijry(r)+ 
lxy(g)+ Ijcy(^> where r, g and b refer to the RGB channels of the image. An example of a 
1 0 Structure Analyzing Function would be A(I, x, y) = - 1 (x+\) y (r)- Note: These three 

categories of Analyzing Functions are not disjoint. For example, the function A(Ijc,y) = I xy ( r ) - 
I(x+i)(y-2)(g) + x is a Color Analyzing Function, a Structure Analyzing Function, and a Location 
Analyzing Function simultaneously. 

"Normalizing" the Analyzing Functions and limiting the range of possible values 
1 5 such that their results have approximately the range of 0 ... 100 will simplify the process. 

The fifth element D in Equation 1 is a "Difference Function" which can compare two 
vectors of n values against each other and provides a single number that is larger the more the 
two vectors of n values differ and zero if the two sets of n numbers are identical. In doing so, 
the function D is capable of weighing each individual number of the two sets with a weight 
20 vector (wj ...„) as in Equation [5]. 

d=D((a l J,(b l J,(w l J) [5] 

D is defined as follows: 

D((a,..n),(bi..n),(w 1 .. n )) = || (ai*w,-b,*w,), (a n *w n - b n *w n ) || [6] 
where || • || refers to any norm, such as the distance in Euclidian space, which is also known 
25 as||.|| 2 . 

In other words, the more ai...„ and b\... n differ, the higher the result of the Difference 
Function Z), while the weights w\ mmM control the importance of each element of the vectors of a 
and b. By setting elements of w to zero, D will disregard the according elements of a and b. 

Suitable Difference Functions in this implementation are: 
30 D (K..„), (6,...„), (w,...„)) = \a r bj\*wj + \a 2 -b 2 \ *w 2 + ... + \a n -b n \ *w„ [7] 

D ((*!...„), (6,...„), (W|„J) 2 = (a / *Wy-6 / *Wy) 2 + ... +(d n *W n -b n * W n f [8] 

The weighed Pythagoras function [8] leads to better results than the simple function 
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[7], while function [8] provides for accelerated processing. To those skilled in the art, the 

norms used in [7] and [8] may also be known as || • || i and || • || 2. 

A function D* that is derived from the function D is defined as follows: 
D*((a x .. n ), (6l...„), (W1..J, g*) = D ((*!...„), (&,...„), (w u .n)) + g* [9] 
5 In other words: D* measures the difference of au n and b\.„ n9 weighed with w\„ M , and 

adds the real number g* to the result. 

For accelerated performance or for simpler implementation, another Difference 

Function D or D * can be made use of which does not utilize weights. Systems as described 
in this disclosure that do not utilize weights are easier to use and faster to compute, but less 

10 flexible. D and D * are defined as follows: 

£((a,...*), (61...*)) = D ((a Un ), (1,1,...,1)) [10] 

D\a x ...*), g) = D* ((*,..„), (6,..„), (1,1,.. .,1), g*) [11] 

The sixth element, V, in Equation 1 is an "Inversion Function" that has the following 

characteristics with V : -> M + : 

15 F(jc)>0forallx >0 

V(y)< V(x)fora\lx<yfovMx,y >0 
Urn x -» 00 of V(x) = 0. 
The Gaussian bell curve or V(x) = l/(x+0.001) are such functions. Note: V{x) = l/x is 
not appropriate as the result of V(0) would not be defined. 
20 In one preferred embodiment, the function in Equation [12] is used, where t is any 

number that is approximately between 1 and 1000. The value t = 50 is a good value to start 
with, if the Analyzing Functions are normalized to a range of 0. .. 100 as referred to in the 
section on "Normalizing Analyzing Functions" that follows equation [4] in this disclosure. 

V(x) = 0.5 (x/ ° [12] 
25 The Inversion function will be used later in this disclosure to calculate an "Inverse 

Difference" between two tuples a\„ n and b\.„ n by calculating V(D*((a\.„ n ) 9 (b\... n )i (wi...„), g*) 
) or V(D ((*,...„), (*,...„), (w,...„)) ) or V(D *((*!...„), (6,.„ n ), g*) ) or V(D ((a,. ..„), .„)) ). The 
purpose of this Inverse Difference is to provide a high value if similarity between the tuples 
a\„ n and b\. mH is detected, and a low value, if the tuples a\„ n and b\ mnH are different. 
30 The seventh element, Ci... m , in equation [1] is a set of m "Controlling Functions". 

Each of those Controlling Functions has m parameters and needs to suit the following 
conditions: 
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Ci(pi...p m ) >0 for all pi... p m and for all 1 <z <m (all pi...p m will never be negative). 
C,{pi.-Pm) is high if /?, has a high value compared to the mean of pi...p m 
Ci(pi...p m ) is low if pi has a low value compared to the mean of pi...p m 
C,+C 2 +...+C m is always 1. 
5 C,{pi...p m ) = Cn(i)(p n(\)-p n(m)) with II being any permutation II:(1 . ..my* (11(1). . . 

n(m)). 

A recommended equation for such a controlling function would be as shown in 
Equation [13]. 

Ctipi. . .p m ) = pi /{pi +/?2+. • • +p m ) [13] 

10 The purpose of a controlling function C, is to provide a large number (close to 1) if 

the f h element of the parameters pi...p m is high relative to the other parameters, and a small 
value (close to 0) if the t h element of the parameters pi...p m is relatively low, and to "down- 
scale" a tuple of m elements so that their sum is 1 .0, while the relations between the elements 
of the m-tuple are constrained. If the Controlling Functions are applied to a set of m Inverse 

1 5 Differences, the m results of the Controlling Functions will be referred to as "Controlled 
Inverse Differences" later in this disclosure. 
Setting the elements F, R and A 

The following section describes the manner in which the user-defined (or otherwise 
defined) m IRPs can be converted into its associated Performing Functions F and tuples R. 

20 Note: In contrary to F and /?, the last four elements of the tuple {A, D, V f Q are 

functions that are defined by the programmer when a system using IRPs is created, and are 
predefined or only slightly adjustable by the user. However, to a certain extent, a system may 
give the user control over the functions A, D, Fand C; and there can be certain components 
of the first two elements Fj. .. w and Rj ... m that will be set by the application without user 

25 influence. This will become clearer later in this disclosure. 

Figure 4 provides a sample image of an image in a dialog box an image processing 
program, for the purposes of illustrating modifications to an image using the current 
invention. For example, graphic tag 30 representing IRP R\ in Figure 4, placed on the left 
apple, will be to increase saturation, graphic tag 32 representing IRP R 2 , placed on the right 

30 apple, will be decrease saturation, and graphic tag 34 representing IRP R^ placed on the sky, 
will darken its associated image component. 

To do so, three performing functions Fy... F3 are necessary, where F/ increases the 



Atty. Docket 13886-2 Express Mail EV 390739541 US 

16 

saturation, F2 decreases the saturation, and F3 is an image darkening image modification. 

The system should typically allow the user to set such a Performing Function before 
or after the user places an IRP in the image. In such cases, the user first defines the type of 
the performing function (such as "sharpen," or "darken," or "increase saturation," etc.) and 
5 then the user defines the behavior of the function (such as "sharpen to 100%," or "darken by 
30 levels," etc.). 

In the current example, three tuples R\...Rs are necessary. For each IRP, there is 
always one tuple R and one Performing Function F. It is not necessary, however, that all 
Performing Functions are different. As previously disclosed, IRPs in the current invention 
10 are used to store Characteristics of an individual pixel or a particular area in an image. As 
such, using the current example of modifying Figure 4, three IRPs are necessary: an IRP that 
stores the Characteristics of the first apple, an IRP that stores the Characteristics of the 
second apple, and an IRP that stores the Characteristics for the sky. 

This can typically be done by reading the Characteristics of the image location where 
1 5 the user has placed an IRP. If a user has placed an IRP on the image coordinate location x, y 
in the image I, the values of R = ((g\.-.gn), g*> (h>i...w„)) can be calculated as follows: 

g/...g„ = Aj{I f x >y ) ... A n (lx t y) [14] 
g* = 0 

wj...w n = default value, for example, 1. 
20 The user may have control over the values of R after they were initially filled. This 

control may be allowed to varying extents, such as weights only versus all variables. 

In our example the two red apples will be modified differently. Presumably, both 
apples have the same color and the same structure, and each only differs in its location. The 
sky, containing the third IRP, has a different location than the apples, and also a different 
25 color. 

As we now see that both location and color are relevant for differentiating between 
the three relevant image areas, it will be obvious that what is needed is at least one or more 
Location Analyzing Functions and one or more Color Analyzing Functions. In cases where 
the application allows the user only to perform global color changes, it would be sufficient to 
30 choose only Color Analyzing Functions. 

Some Analyzing Functions are as follows, where I^j refers to the red channel's 
value of the image I at the location x,y and so forth. 

Ai{lx,y)=x [15a] 
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A 2 (l,x,y) = y 


[15b] 


A 3 (I,x,y) = l^r) 


[15c] 


A 4 (lx,y) = IxKg) 


[15d] 


A 5 (I,x,y) = ljtfb) 


[15e] 



5 A j and A2 are Location Analyzing Functions and A 3 through A 5 are Color Analyzing 

Functions. 

Note: A3 through A 5i which only provide the red, green, and blue values, are suitable 
functions for a set of color-dependent analytical functions. For even better performance it is 
recommended to derive functions that calculate luminosity, saturation, etc., independently. 
10 Using the channels of the image in Lab color mode is appropriate. However, the following 
Analyzing Functions are also examples of appropriate Analyzing Functions, where the 
capitalized variables^ Y, R, G, B represent the maximum possible values for the coordinates 
or the color channels. 

Aj(I,x,y)=x* 100/X [16a] 
15 A&x,y)=y* 100/7 [16b] 

A 3 (I,x f y) = 0W,+ Ion + 1^) ) * 1 00 / (R + G + B) [ 1 6c] 

A 4 &x,y) = 100 * (I^ (r) - 1^) ) / (R+G) + 50 [16d] 
A 5 (L t x,y) = 100 * (I xyir) - I xy(b) ) / (R+B) + 50 [16e] 
Equations [16] shows Analyzing Functions that are also normalized to a range of 
20 0... 100 (see the description for normalizing Analyzing Functions after equation [4]) 

Normalizing the Analyzing Functions aids in the implementation, as normalized Analyzing 
Functions have the advantage that their results always have the same range, regardless of the 
image size or other image characteristics. The Analyzing Functions found in Equations [15] 
will be used throughout this disclosure when discussing values from Rj ,.. m 
25 Note: It may not be useful to adjust the set of Analyzing Functions from image to 

image. It may be preferable to use one set of Analyzing Functions that is suitable for many or 
all image types. When the current invention is used for standard color enhancements, the 
Analyzing Functions of Equations [16] are good to start with. 
A Closer Look at IRPs 
30 As previously discussed in this disclosure, the tuples R of an IRP store the 

information of the Characteristics of the region to which an operation will be applied, the 
region of interest. These tuples R acquire the Characteristics typically by applying the n 
analytical functions to the image location 1^ where the IRP was placed, as in equation [14]. 
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In the current embodiment, the Difference Function D* will compare the values g/...g„ 
of each IRP to the results of the n Analyzing Functions for all pixels in the image, using the 
weights w/...w n . 

For example, if the pixel in the middle of the left apple has the coordinates (10, 100) 
5 and the RGB color 1 50, 50, 50 (red), then the Analyzing Functions Aj,..A n of this pixel will 
have the values ^i=10, ^ 2 =100, ^ 3 =150, ^ 4 =50, ^ 5 =50, therefore, the values g/...g„ will be 
set to (10, 10, 150, 50, 50). 

g* is set to zero for this IRP. 

The weights will control the significance of the individual elements of gj...gs. See 
10 Equations [6], [7] and [8]. For example, if the weights w/...w 5 are set to (10,10,1,1,1), the 

location related information, gained through A\ and A2, will be more significant than the color 
related information from A3 through A 5 . (This IRP would be more location dependent than 
color dependent). 

If, however, w/...w 5 = (0,0,3,3,0) is set, only the red and green channels of the pixel 
15 information would be considered by the Difference Function, and the IRP would not 

differentiate between the location of a pixel or its blue channel. As previously mentioned, in 
Figure 4 the location-dependent and color-dependent Characteristics play a role in 
differentiating the apples from each other and from the sky. Therefore, we will use equal 
weights for all 5 characteristics. 
20 Setting all weights all to 1 , the first IRP would be: 

R I = (gl...g5,g*wl...w5) = ((10,100,150,50,50), 0, (1,1,1,1,1)) 

(the first apple at the coordinate 10,100 with the color 150,50,50) 

The second and third IRP could have values such as 

R 2 = ((190,100,150,50,50), 0, (1,1,1,1,1)) 
25 (the second apple at the coordinate 190,100 with the color 150,50,50) 

R 3 = ((100,10,80,80,200), 0, (1,1,1,1,1)). 

(the sky at the coordinate 100,10 with the color 80,80,200) 
The mixing function 

An abbreviation related to the Difference Function follows. The purpose of the 
30 Difference Function is to calculate a value that indicates how "different" a pixel in the image 
is from the Characteristics that a certain IRP is associated with. 

The "Difference" between an IRP R = ((g/...g„), g*,(w/... w„)) and a pixel 1^ can be 
written as follows: 
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\R-I,y\= D*((g,...g n ), (A,(L,x,y), ... , A n (l,x,y)), (w,, w n ), g* ) [17] 

The Difference referred to in this embodiment is always the result of the Difference 

function, and should not be confused with the "spatial" distance between two pixels in an 

image. 

5 If, for ease of implementation or for faster computing of the Mixing Function, the 

Difference Functions D, D or D* are used, the abbreviation would be: 

\R-l xy \= D((g,... gn ), (A,(I,x,y), ... , A„(I,x,y)), (w,, w„ ) ) [18] 
\R-l xy \ = D ((g,...g„), (A,(l,x,y), ... , A n (I,x,y))) [19] 
\R-I xy \ = D \(g,...g n ), (A,(l,x,y), ... , A„(lx,y)), g* ) [20] 
1 0 Given the 7-tupel of an IRP based image modification (F/...M, Ri.. KQ 

then the modified image l*xy is as show in Equation [21]. 

i; =S^(I^.^)*C j (K(|i? 1 -I^|),...,F(|i? m -I ;[y |)) [21] 

1=1 

Apply this equation to each pixel in the image I, to receive the processed image I*, 
where all Performing Functions were applied to the image according to the IRPs that the user 
1 5 has set. This equation compares the n Characteristics of each pixel x, y against all IRPs, and 
applies those Performing Functions F { to a greater extent to the pixel, whose IRPs have 
similar Characteristics, while the Controlling Function ensures that the sum of all functions F t 
does not exceed unwanted ranges. 

In an even further preferred embodiment of the current invention, equation [22] would 
20 be used. 

In contrast to equation [21], equation [22] requires that the Inversion Function Kdoes 
not exceed values of approximately 1. The Gaussian Bell curve V(x) = e" x or l/(x+l) or 
equation [12] could be such functions. The function AF expresses the difference between the 
25 original and modified image (where I \ y = I ^ + AF(I, x, y) instead of I ^ = x, y\ see 
Equation 2). 

When comparing Equation [21] and [22], the terms V(\R r Jxy\) represent the Inverse 
Difference of the currently processed tuple Ri and the pixel 1^. Only equation [21] uses 
Controlled Inverse Differences. If equation [21] is used, each pixel in the image will be 
30 filtered with a 100% mix of all Performing Functions, regardless if an image region contains 
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a large or a small number of IRPs. The more IRPs that are positioned in the image, the less 
effect an individual IRP will have if Equation [21] is used. If Equation [22] is used, the IRPs 
will not show this competitive nature. That is, each IRP will modify the image to a certain 
extent regardless whether it is placed amidst many other IRPs or not. Therefore, if Equation 
5 [22] is used, placing multiple IRPs in an image area will increase the total amount of image 
modification in this area. 

Further Embodiments 

In a further embodiment, the concept of "Abstract IRPs" can be used to enhance the 
performed image modification, or to change the behavior of the image modification. 

10 Abstract IRPs are similar to other IRPs as they are pairs of a Performing Function F 

and a set of values R. Both Abstract IRPs and IRPs may be used together to modify an 
image. Abstract IRPs, however, are not "user defined" IRPs or IRPs that are placed in the 
image by the user. The function of an Abstract IRP can be to limit the local "effect" or 
intensity of an IRP. In this regard, Abstract IRPs are typically not "local", i.e., they affect the 

1 5 entire image. Abstract IRPs can be implemented in a manner that the user turns a user- 
controlled element on or off as illustrated later, so that the Abstract IRPs are not presented as 
IRPs to the user, as shown in Figure 4. 

Note: The use of Abstract IRPs as disclosed below requires that equation [21] is 
implemented as the mixing function, and that the Difference function is implemented as 

20 shown in equation [17] or [20]. 

In Figure 4 the user has positioned graphic tags 30, 32, and 34 representing IRPs 
R\ . . .i?3. Controls 36, 38, and 40 indicate a set of three possible user controls. When control 
36 is used, the application would use one additional pre-defined Abstract IRP in the image 
modification. Such pre-defined, Abstract IRPs could, for example, be IRPs R4 through R 6 as 

25 described below. 

When the check box in control 36 is enabled, Abstract IRP R 4f is utilized. Without the 
use of an Abstract IRP, when an image has an area such as the cactus 42 which is free of 
IRPs, this area will still be filtered by a 100% mix of the effects of all IRPs (see equation [19] 
and the Controlling Function Q. In the current image example, the cactus 42 would be 

30 affected by a mix of the IRPs R1...R3, although the user has placed no IRP on the cactus. 

To remedy this, Abstract IRP R4 is utilized which makes use of the g* value. Note: 
g* is used as described below when the mixing function of equation [21] is being 
implemented. 
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The Abstract IRP could have zero weights and a g* value greater than zero, such as 
^ = ((0,0,0,0,0), 50, (0,0,0,0,0)) 

The Difference Function |i? 4 -Ijcy| will return nothing but 50 whatever the 
Characteristics of the pixel 1^ might be. The value of g* should be in the range of 1 to 1000. 
5 50 is a good value to start with. 

The purpose of this IRP and its R 4 is that pixels in areas free of IRPs, such as in the 
middle of the cactus 42, will have a lower Difference to R 4 (which is constantly set to 50) 
than to RJ...R3. For pixels in image areas where one or more IRPs are set, R 4 will not be the 
IRP with the lowest Difference, as a different IRP will likely have a lower Difference. In 
10 other words: areas free of non- Abstract IRPs are controlled predominantly by R 4 > and areas 
that do contain non- Abstract IRPs will be affected to a lesser extent by R 4 . If the Performing 
Function F 4 is set to a function that does not change the image (F41yX,y) = lxy\ R 4 ensures 
that areas free of IRPs will remain mainly unaffected. 

In order to make Abstract IRP R 4 more effective (i.e., IRPs R/..R3 less effective), g* 
15 can be lowered, and the value g* in R 4 can be raised to make the "active" IRPs /?/..i?jmore 
effective. A fixed value for g* in R 4 may be implemented if the system that is programmed is 
designed for image retouchers with average skills for example, and applications designed for 
advanced users may permit the user to change the setting of g*. 

In an even further embodiment of the current invention, Abstract IRPs could be used 
20 whereby an IRP has weights equaling zero for the location dependent parameters, and values 
for gj...g n which would represent either black or white, combined with a Performing Function 
which does not affect the image. 

Two of such Abstract IRPs - one for black, one for white - would be suitable to 
ensure that black and white remain unaffected. Such Abstract IRPs could be: 
25 R s = ( (0,0,255,255,255), 0, (0,0,1,1,1) ) 

* 6 = ( (0,0,0,0,0), 0, (0,0,1,1,1)) 
As with R4 and F4, the Performing Functions F 5 and F6 would also be functions that 
do not perform any image modification, so the IRPs 5 and 6 would ensure that colors such as 
black and white remain mainly unaffected by the IRPs that the user places. 
30 As shown in control 38 and control 40, these Abstract IRPs can be implemented 

providing the user with the ability to turn checkboxes or similar user controls on or off. Such 
checkboxes control the specified function that the Abstract IRPs would have on the image. 
When the associated checkbox is turned on, the application uses this Abstract IRP This 
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process is referred to as "load abstract IRPs" in step 18 of Figure 3. 

It is not necessary that all Abstract IRPs are associated with a Performing Function 
that leaves the image unaffected. If for instance an implementation is programmed that allows 
the user to sharpen the image, an abstract IRP such as R4 above can be implemented, where 
5 the associated Performing Function F4 sharpens the image to 50%. The user could then place 
IRPs whose Performing Functions sharpen the image to for instance to 0%, 25%, 75% or 
100% in the image. This would mean that the image is sharpened to an individual extent 
where the user has set IRPs, and to 50% anywhere else. 

In an even further embodiment, the IRP based image modification can be used in 
10 combination with a further, global image modification I = Af(I, x, y), where Mis an image 
filter, combining the IRP based image modification and the uniform image modification M as 
shown in Equation [23]. 

i; = M(I xy+ f j AF i (I,x,y)*V(\R i -l xy \)) [23] 

1=1 

Equation [23] is derived from equation [21]. Equation [22] could also be utilized for 

1 5 this embodiment. The current embodiment is useful for a variety of image filter types M, 

especially those that lead to unwanted image contrast when applied, causing what is known to 
those skilled in the art as "blown-out areas" of a digital image. Such image filters M could be 
color to black and white conversions, increasing the overall contrast, inverting the image, 
applying a strong stylistic effect, a solarization filter, or other strong image modifications. 

20 Applying such an image modification such as a color to black and white conversion 

without the current invention, the user would first convert the image to black and white, 
inspect areas of the resulting black and white image that are too dark or too bright, then undo 
the image modification, make changes to the original image to compensate for the filter 
application, and then re-apply the image modification, until the resulting image no longer has 

25 the unwanted effects. 

While implementing this filter in combination with an IRP based image modification 
as shown in Equation [23], the user can modify contrast and color of the image as the image 
modification M is applied, such as in the example of the black and white conversion, thus 
accelerating the method of application by the user for the black and white conversion process 

30 and providing improved results. 

In an even further embodiment, the Performing Functions F x can be replaced with 
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"Offset Vectors" S, = (ax. Ay.) T , where 5/... ro are the m Offset Vectors associated with the m 

IRPs, and Ax and Ay are any real numbers. In this case, the user would define such an Offset 
Vector of an IRP for instance by defining a direction and a length, or by dragging an IRP 
symbol with a mouse button different from the standard mouse button. The mixing function, 
5 for instance if derived from equation [21], would then be 

m 

K = 2>, *c,n\ R, -/„ K -i v I)) p4] 

1=1 

Of course, as the result of this function is assembled of vectors of R 2 , the result is a 
matrix S xy of the same horizontal and vertical dimensions as the image, whose elements are 
vectors with two elements. For further reference, I refer to this matrix as an "Offset Matrix". 
10 Using this implementation, the user can easily attach IRPs to regions in the image and 

at the same time define in which directions the user wants these regions to be distorted or 
altered. 

The result of the mixing function is an offset matrix that contains information relating 
to in which direction a pixel of the original image I needs to be distorted to achieve the 
1 5 distorted image I d . The benefit of calculating the Offset Matrix this way is that the Offset 

Matrix adapts to the features of the image, provided that the vectors R\... m have weights other 
than zero for pixel luminosity, chrominance, and structure Characteristics. The image I d can 
be calculated the following way: 

(1) Reserve some memory space for I d , and flag all of its pixels. 
20 (2) Select the first coordinate (x,y) in I. 

(3) Write the values (such as r,g,b) of the pixel I xy into the picture I d at the 
location (x,y) + S xy , and un-flag the pixel at that location in f 1 . 

(4) Unless all pixels in I are considered, select next coordinate (x,y) and proceed 
with step (3); 

25 (5) Select first pixel in I d that is still flagged. 

(6) Assign the values (such as r,g,b) of the closest non-flagged pixel to this pixel. 
If multiple non-flagged pixels are equally close, select the values of that pixel that 
was created using the lowest Offset Vector S xy . 

(7) If flagged pixels are left, select next flagged pixel in I d and proceed with step 
30 (6). 

In other words, copy each pixel from I into I* 1 while using the elements of the Offset 
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Matrix S for offsetting that pixel. Those areas that remain empty in J 6 shall be filled with the 
pixel values neighbored to the empty area in I d , while values of pixels that were moved to the 
least extent during the copy process shall be preferred. 

In a further embodiment, a plurality or IRPs can be saved and applied to one or more 
5 . different images. In batch processing applications, this plurality of IRPs can be stored and 
applied to multiple images. In such an embodiment, it is important that IRPs whose weights 
for location-dependent characteristics 'are zero. 

In a further embodiment, the user may be provided with simplified control over the 
weights of an IRP by using a unified control element. In Equations [15] and Equations [16], 

10 five Characteristics are utilized, two of which are location dependent Characteristics sourcing 
from Location Analyzing Functions. 

In creating such a unified control element, one control element controls these two 
weights. This unified control element could be labeled "location weight," instead of the two 
elements "horizontal location weight" and "vertical location weight." 

15 In a further embodiment, user control elements may be implemented that display 

different values for the weights as textual descriptions instead of numbers, as such numbers 
are often confusing to users. Those skilled in the art will recognize that it may be confusing to 
users that low values for weights lead to IRPs that have more influence on the image, and vice 
versa. Regarding weights for location-dependent Characteristics (such as w/ and w 2 in the 

20 current example), the user could be allowed to choose one out of five pre-defined weights for 
textual descriptions of different values for the location dependent weights wi and w 2 as show 
in Table 1 . 



Table 1 










"global" 


0 


0 


"almost global" 


0.3 


0.3 


"default" 


1 


1 


"local" 


3 


3 


"very local" 


8 


8 



Figure 5 illustrates how such simplified user control over weights may be 
25 implemented in an image processing program. 

In a further embodiment, the user control over weights could be simplified to such an 
extent that there are only two types of weights for IRPs that the user can choose from: 
"strong" that utilizes weight vectors such as (1,1,1,1,1) and "weak" that utilizes weight 
vectors such as (3,3,3,3,3). Note: As mentioned before, large weights make the area that an 
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IRP has influence on smaller, and vice versa. 

For example, the user may place IRPs in the sky with an associated enhancement to 
increase the saturation of an area identified by one or more IRPs. In the same image, the user 
may place additional IRPs with an assigned function to decrease contrast, identifying changes 
5 in contrast and the desired changes in contrast based on the location of each individual IRP. 
In a preferred embodiment, IRPs may include a function that weights the intensity of the 
image-editing function as indicated by the user. 

In a different implementation of the invention, IRPs could be placed to identify a 
color globally across the image, and using an associated command, increase the saturation of 
10 the identified color. 

In a still further preferred embodiment, IRPs could be used to provide varying degrees 
of sharpening across a digital image. In such an implementation, multiple IRP's could be 
placed within specific image regions or image characteristics, such as the eyes, the skin, and 
hair of a portrait, and different sharpening intensities assigned to each IRP and applied to the 
15 digital image while considering the presence of color and/or contrast and the relative 
difference of each IRP from one another to provide the desired image adjustment. 

All features disclosed in the specification, including the claims, abstract, and 
drawings, and all the steps in any method or process disclosed, may be combined in any 
combination, except combinations where at least some of such features and/or steps are 
20 mutually exclusive. Each feature disclosed in the specification, including the claims, 

abstract, and drawings, can be replaced by alternative features serving the same, equivalent or 
similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, 
each feature disclosed is one example only of a generic series of equivalent or similar 
features. 

25 This invention is not limited to particular hardware described herein, and any 

hardware presently existing or developed in the future that permits processing of digital 
images using the method disclosed can be used, including for example, a digital camera 
system. 

A computer readable medium is provided having contents for causing a computer- 
30 based information handling system to perform the steps described herein, and to display the 
application program interface disclosed herein. 

The term memory block refers to any possible computer-related image storage structure 
known to those skilled in the art, including but not limited to RAM, Processor Cache, Hard 
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Drive, or combinations of those, including dynamic memory structures. Preferably, the 
methods and application program interface disclosed will be embodied in a computer 
program (not shown) either by coding in a high level language, or by preparing a filter which 
is complied and available as an adjunct to an image processing program. For example, in a 
5 preferred embodiment, the methods and application program interface is compiled into a 
plug-in filter that can operate within third party image processing programs such as Adobe 
Photoshop®. 

Any currently existing or future developed computer readable medium suitable for 
storing data can be used to store the programs embodying the afore-described interface, 

10 methods and algorithms, including, but not limited to hard drives, floppy disks, digital tape, 
flash cards, compact discs, and DVDs. The computer readable medium can comprise more 
than one device, such as two linked hard drives. This invention is not limited to the particular 
hardware used herein, and any hardware presently existing or developed in the future that 
permits image processing can be used. 

15 Any currently existing or future developed computer readable medium suitable for 

storing data can be used, including, but not limited to hard drives, floppy disks, digital tape, 
flash cards, compact discs, and DVDs. The computer readable medium can comprise more 
than one device, such as two linked hard drives, in communication with the processor. 

A method for image processing of a digital image has disclosed comprising the steps 

20 of determining one or more sets of pixel characteristics; determining for each pixel 

characteristic set, an image editing function; providing a mixing function algorithm embodied 
on a computer-readable medium for modifying the digital image; and processing the digital 
image by applying the mixing function algorithm based on the one or more pixel 
characteristic sets and determined image editing functions. In one embodiment, the mixing 

25 function algorithm comprises a difference function. Optionally, the difference function 

algorithm calculates a value based on the difference of between pixel characteristics and one 
of the one or more determined pixel characteristic sets. In another embodiment, the mixing 
function algorithm includes a controlling function for normalizing the calculations. 

In a further embodiment, the method adds the step of determining for each pixel 

30 characteristic set, a set of weighting values, and the processing step further comprises 
applying the mixing function algorithm based on the determined weighting value set. 

In a further embodiment, a first pixel characteristic set is determined, and at least one 
characteristic in the first pixel characteristic set is location dependent, and at least one 
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characteristic in the first pixel characteristic set is either color dependent, or structure 
dependent, or both. Alternatively, a first pixel characteristic set is determined, and at least 
two different characteristics in the first pixel characteristic set are from the group consisting 
of location dependent, color dependent, and structure dependent. 
5 A method for processing of a digital image has been disclosed, comprising the steps 

of receiving the coordinates of one or more than one image reference point defined by a user 
within the digital image; receiving one or more than one image editing function assigned by 
the user and associated with the coordinates of the one or more than one defined image 
reference point; providing a mixing function algorithm embodied on a computer-readable 
10 medium for modifying the digital image; and processing the digital image by applying the 

mixing function algorithm based on the one or more than one assigned image editing function 
and the coordinates of the one or more than one defined image reference point. The method 
may optionally further comprise displaying a graphical icon at the coordinates of a defined 
image reference point. 

1 5 A mixing function algorithm suitable to the invention has been described, and 

exemplar alternative embodiments are disclosed, including a group consisting of a Pythagoras 
distance approach which calculates a geometric distance between each pixel of the digital 
image to the coordinates of the one or more than one defined image reference point, a color 
curves approach, a segmentation approach, a classification approach, an expanding areas 

20 approach, and an offset vector approach. Optionally, the segmentation approach comprises 
multiple segmentation, and additionally optionally the classification approach adjusts for 
similarity of pixel attributes. The mixing function algorithm may optionally operate as a 
function of the calculated geometric distance from each pixel of the digital image to the 
coordinates of the defined image reference points. 

25 Optionally, the disclosed method further comprises receiving one or more assigned 

image characteristics associated with the coordinates of a defined image reference point, and 
wherein the mixing function algorithm calculates a characteristic difference between the 
image characteristics of a pixel of the digital image and the assigned image characteristics. 
The mixing function algorithm may also calculate a characteristic difference between the 

30 image characteristics of a pixel and the image characteristics of one or more pixels 
neighboring the coordinates of one or more defined image reference point. 

Additionally, optionally other steps may be added to the method. For example, the 
method may further comprise receiving one or more weighting values, and the processing 
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step farther comprising applying the mixing function algorithm based on weighting values; or 
further comprise receiving one or more regions of interest associated with the coordinates of 
one or more defined image reference point; or further comprise the step of providing an 
application program interface comprising a first interface to receive the coordinates of the 
5 one or more defined image reference points, and a second interface to receive the one or more 
assigned image editing functions. 

A method for processing of a digital image comprising pixels having image 
characteristics has been disclosed comprising the steps defining the location of image 
reference points within the digital image; determining image editing functions; and 

10 processing the digital image by applying the determined image editing functions based upon 
either the location of the defined image reference points, or the image characteristics of the 
pixels at the location of the defined image reference points, or both. 

A method for image processing of a digital image has also been disclosed comprising 
the steps of providing one or more than one image processing filter; setting the coordinates of 

1 5 one or more than one image reference point within the digital image; providing a mixing 
function algorithm embodied on a computer-readable medium for modifying the digital 
image; and processing the digital image by applying the mixing algorithm based on the one 
or more than one image processing filter and the coordinates of the one or more than one set 
image reference point. Optionally, various filters may be used, including but not limited to a 

20 noise reduction filter, a sharpening filter, or a color change filter. 

Also, any element in a claim that does not explicitly state "means for" performing a 
specified function or "step for" performing a specified function, should not be interpreted as a 
"means" or "step" clause as specified in 35 U.S.C. § 112. 



